<html>
<head><meta charset="utf-8"><title>Snapshot testing rustc? · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html">Snapshot testing rustc?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="207537330"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537330" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537330">(Aug 20 2020 at 15:40)</a>:</h4>
<p>Do we have a library for snapshot testing somewhere in rustc?</p>



<a name="207537356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537356">(Aug 20 2020 at 15:41)</a>:</h4>
<p>Specifically, for writing snapshot <strong>unit</strong> tests?</p>



<a name="207537366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537366">(Aug 20 2020 at 15:41)</a>:</h4>
<p>Something like this: <a href="https://github.com/rust-analyzer/rust-analyzer/pull/5101">https://github.com/rust-analyzer/rust-analyzer/pull/5101</a></p>



<a name="207537517"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537517" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537517">(Aug 20 2020 at 15:42)</a>:</h4>
<p>no, I don't think so</p>



<a name="207537527"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537527" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537527">(Aug 20 2020 at 15:42)</a>:</h4>
<p>Context: I am rewriting rustdoc syntax highlighting (long yak). It currently is tested via assert_eq, which is painful to modify by hand: </p>
<p><a href="https://github.com/rust-lang/rust/blob/5fff3824bc77cd7052e2515f82259e32f8ad1486/src/librustdoc/html/highlight/tests.rs#L25-L82">https://github.com/rust-lang/rust/blob/5fff3824bc77cd7052e2515f82259e32f8ad1486/src/librustdoc/html/highlight/tests.rs#L25-L82</a></p>



<a name="207537544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537544">(Aug 20 2020 at 15:42)</a>:</h4>
<p>I think we just use UI tests for this</p>



<a name="207537581"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537581" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537581">(Aug 20 2020 at 15:43)</a>:</h4>
<p>with println!/eprintln! and compare-run-output or whatever the flag is</p>



<a name="207537776"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537776" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537776">(Aug 20 2020 at 15:45)</a>:</h4>
<p>Uhu... rustdoc's integration tests are based on <code>htmldocck</code> which isn't exactly a UI test... And UI tests need special harness for running. </p>
<p>I guess, what I am trying to say is that we've been using expect <em>unit</em> tests with a great success in rust-analyzer, and I wonder if we should add them to rustc?</p>



<a name="207537805"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537805" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537805">(Aug 20 2020 at 15:45)</a>:</h4>
<p>I'd not be opposed, we just have almost zero unit tests in rustc</p>



<a name="207537823"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537823" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537823">(Aug 20 2020 at 15:45)</a>:</h4>
<p>but seems good to add them where possible!</p>



<a name="207537927"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537927" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537927">(Aug 20 2020 at 15:46)</a>:</h4>
<p>Are there any specific use-cases where we want them?</p>



<a name="207537960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207537960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207537960">(Aug 20 2020 at 15:46)</a>:</h4>
<p>If we  <em>just</em> add a library, it'll just die without much use I am afraid.</p>



<a name="207538021"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538021" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538021">(Aug 20 2020 at 15:47)</a>:</h4>
<p>Many parts of the compiler don't lend themselves well to unit testing today (too much global state)</p>



<a name="207538038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538038">(Aug 20 2020 at 15:47)</a>:</h4>
<p>or, you know, requiring a ton of setup</p>



<a name="207538084"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538084" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538084">(Aug 20 2020 at 15:47)</a>:</h4>
<p>but for things like lexer and rustdoc's syntax highlighting it seems like neither is probably true</p>



<a name="207538210"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538210" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538210">(Aug 20 2020 at 15:48)</a>:</h4>
<p>Even more specifically -- I am working exactly on purging the global state away, which makes it possible to write unit tests. But I lack convenient way to do that.</p>



<a name="207538251"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538251" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538251">(Aug 20 2020 at 15:48)</a>:</h4>
<p>assert_eq unit tests are just a pain to maintain long-term</p>



<a name="207538286"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538286" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538286">(Aug 20 2020 at 15:48)</a>:</h4>
<p>I guess, I should file an MCP about adopting something along the lines of <code>expect!</code> from rust-analyzer?</p>



<a name="207538530"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538530" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538530">(Aug 20 2020 at 15:50)</a>:</h4>
<p>I guess I don't see why you don't want to just add it?</p>



<a name="207538539"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538539" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538539">(Aug 20 2020 at 15:50)</a>:</h4>
<p>like, I don't think it needs an MCP</p>



<a name="207538591"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538591" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538591">(Aug 20 2020 at 15:51)</a>:</h4>
<p>even if it doesn't get a lot of use initially, seems like a good thing to have that doesn't really hurt us any</p>



<a name="207538908"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207538908" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207538908">(Aug 20 2020 at 15:53)</a>:</h4>
<p>There's some design-space around the library for snapshot testing. The thing written for rust-analyzer is something maximally simple I came up with, as an alternative to more feature-full stuff from <a href="http://crates.io">crates.io</a> (insta, k9). </p>
<p>I obviously like my thing more, but I don't want to "push" rustc to it without consensus. I guess, sending a PR which shows the thing wont' hurt.</p>



<a name="207539292"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207539292" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207539292">(Aug 20 2020 at 15:56)</a>:</h4>
<p>I think starting simple and with something we're already "using" makes sense</p>



<a name="207629176"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Snapshot%20testing%20rustc%3F/near/207629176" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Snapshot.20testing.20rustc.3F.html#207629176">(Aug 21 2020 at 12:40)</a>:</h4>
<p>PR's up: <a href="https://github.com/rust-lang/rust/pull/75773">https://github.com/rust-lang/rust/pull/75773</a></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>